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DIGITAL MUSICAL INSTRUMENT SYSTEM 
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15 



20 



The present invention relates to an interactive multimedia apparatus. 

The present invention is an improvement to invention disclosed in International publication 
number WO 01/95052 by the same inventor, the contents of which are incorporated herein 
by reference thereto. 

It is an object of the present invention to provide an interactive multimedia apparatus 
which will provide users with a digital instrument platform allowing a wider range of more 
challenging experiences. Additionally, it presents users with a sophisticated digital 
musical instrument which will enable them to create musical and other multimedia content 
as if they were accomplished musicians/artists. Additionally, it is an object of the 
invention to allow non-musical trained users to compose, accompany, solo, gig and have 
fim as if they were an accomplished instrument player. Finally, it is an object of the 
present invention to assist users in developing knowledge of chord structures, complex 
chord structures, voicing, scales and the fingering of notes on the firet of stringed 
instruments. 

Accordingly, the present invention provides an interactive multimedia apparatus 
comprising:- 

a digital musical instrument having a pluraUty of control members including at least 
one limited state (on/off) control member and at least dynamic range state control member; 

a central control xmit having a store of digital media stored thereon and a suite of 
software for interpreting the state of the control members in order to select, open and 
render the digital media; 



30 



a control xmit associated with the digital musical instrument having a CPU and a 
sensing means for identifying the state of the control members, the control unit having 
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means for commiinicating the current state of the control members to the central control 
unit; 



a means for communications between the central control unit and the CPU; and 

5 

the sensing means comprising a plurahty of independent transducers each of which 
monitors the activation of a separate dynamic range state control member. 



Advantageously, the provision of an independent transducer to monitor each 
1 0 dynamic range state control member dramatically increases the functionality of the 
apparatus and further increases the interaction between an operator and the apparatus 
allowing an operator greater control over the output generated by the apparatus. 



Preferably, the independent transducers are selected from a group of electrical, 
1 5 optical, pressure, movement, magnetic and piezo-electric transducers. 

All of these types of transducers provide acceptable solutions for the sensing of 
vibrations, and the amplitude of the vibrations. 

20 Ideally, the piezo-electric transducer is selected from a group consisting of piezo- 

electric crystal transducers, piezo-electric ceramic transducers and piezo-electric film 
transducers. 



In a particularly preferred embodiment, the piezo-electric transducer is a piezo- 
25 electric crystal transducer. 



Preferably, the digital musical instrument is a guitar/guitar type device. 



Ideally, a number of the dynamic range state control members are strings of a 
30 stringed instrument. 
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Preferably, each independent transducer monitors one individual string and 
converts movement of the string into electrical signals. 

Ideally, at least one dynamic range state control member is a foot pedal. 

Preferably, one or each foot pedal is monitored by a potentiometer. 

Ideally, activation of the foot pedal dynamically modifies a variable control such as 
volume, pan or special effect parameter controls. 

Preferably, the central control unit has a visual display unit (V.D.U.) and the suite 
of software has a graphical user interface (G.U.I) displayable on the V.D.U.. 



Ideally, the control unit of the digital musical instrument has a control panel 
1 5 comprising a plurality of limited state control members, namely switches for navigating 
through the G.U.L of the software suite and also modifying the parameters of the software 
suite. 



Preferably, the central control unit has memory for storing user assignments of digital 
20 media against specific control members using the control panel, the memory being 
accessible by a software module of the software suite in response to activation of a control 
member or by activation of a combination of one or more limited state or dynamic range 
state control members to open, render, modify, adjust, add effects, change parameters and 
change controls of the rendered digital media. 

25 

Ideally, indicators are provided on the digital musical instrument and are controllable by 
software on the central control unit in response to an assignment of digital media against 
specific control members by a user, the indicators being provided to show the user which 
of the dynanodc range state control members have been assigned to produce an audio/visual 
30 output if activated. 
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Preferably, the indicators are a plurality of light emitting diodes disposed on the digital 
musical instrument below each string, each L.E.D. being associated with one string. 

Ideally, at least one limited state (on/off) control member and at least one dynamic range 
5 state control member are provided. 

Preferably, between one and sixteen limited state (on/off) control members and between 
one and twelve dynamic range state control members are provided. 

10 Ideally, each control member is associatable with any file stored in the store of the central 
control unit. 

Preferably, at least some of the digital audio files are recorded instrument notes fi:om a 
plurality of stringed instrument types. 

15 

Ideally, the store further comprises a chord and scale store of digital media containing the 
note associations for a wide range of defined chord and scale sequences, the store also 
defining the correct note associated with each string position for the selected chord or 
limited scale sequence so that the notes are played in the correct sequence when the strings 
20 are strummed up or down. 

Ideally, the central control unit has input/output connections for the Intemet. 

Preferably, the software in the central control unit detects the length, and amplitude and the 
25 firequency of the electrical signals received from the digital musical instrument in response 
to a control member's excitation and alters its rendering of the digital media mapped to 
that control member or any other control member in sympathy with that excitation. 



30 



The software always generates an output even if the user operates the control members at 
the wrong tune or incorrectly and in which the outputs provided are sympathetic to the 
main track's structure and melody. 
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Ideally, the digital media are rendered simultaneously with a CD, DVD or other primary 
source of music playing on the audio-visual equipment. Soimd is generated in direct 
response to an operator's action, which enhances the overall musical experience for the 
operator. 

5 

Preferably, the apparatus is provided with a series of special effect controls which when 
operated by a user send signals to the software to produce a variety of special effects on the 
digital media output. There are a variety of different ways of manipulating sounds using 
ejffects. Frequencies of samples can be raised or lowered, resonance can be distorted 
1 0 (overdrive) or echo can be added through delay and feedback. 

Conveniently, the apparatus includes means for storing the newly generated output to any 
desired storage device such as a hard disk, a compact disc, a DVD device or the like. 

15 Preferably, tiie apparatus includes a series of visual display screens which are operable to 
enable a user to launch a game experience, select different operating modes, choose a 
source music device, select a backing track and/or assign digital sound effects files to the 
special effects controls on the digital musical instrument. 

20 Optionally, the digital media may contain a riff, a sample, a loop or a track, A riff is a 
series of notes that fonn a section of a musical track. A song might contain a guitar riff of 
eight notes followed by a series of guitar riffs to form a lead solo. Some software music 
studios have riff generators that allow the creation of unique riffs instead of using pre- 
recorded riffs. There are also different digital storage formats for riffs such as .wav and 

25 .mp3. A sample is a pre-recorded piece of music that is usually not very long such as a 
five-second bass riff, or a two-second drum loop. Many CD's are available that offer 
thousands of royalty-free sound samples. A loop is a riff that when repeated over and over 
again forms a seamless track of music. A bass loop may contain a six-note riff that can be 
repeated a number of times to form some of the bass-line of a song. 

30 

Preferably, the apparatus includes an interface for sending the electrical signals generated 
by the transducers to the central control unit, the interface being provided between the 
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transducers and the input port of the central control unit, the interface unit enabling the 
user to generate a plurality of different control signals to the central control unit. 

Ideally, the interface includes an amplifier and level detectors to detect the force with 
5 which the user strums or strikes the strings. The software includes means to decipher the 
electrical signals jfrom the instrument and alter one or more parameters of the rendered 
media in sympathy with the force of the string excitation. 

Ideally, the interface unit is provided with a potentiometer which varies the duration of the 
1 0 sound of the digital media file. 

Ideally, customised driver software is provided with the instrument avoidmg the necessity 
for calibration. 

15 Preferably, the communication means deployed between the digital musical instrument and 
the central control unit is a Universal Serial Bus (USB). 

Preferably, the central control unit comprises a personal computer, a cable or satellite 
television decoder or a games console and the audio/audio-visual means comprises a mono 
20 or stereo high fidelity audio apparatus, a television, a monitor or a like audio/audio-visual 
output means. 

In a still fiirther aspect of the invention, an operator can use the digital musical instrument 
and the software interface as a sixteen or twenty-four track-recording studio. The studio 
25 allows an operator to save their compositions in a format for fixture rendering and also in a 
format for writing their own CD's. Other export formats are MPS and wav. 

Ideally, an operator can drop samples of riffs and loops onto individual tracks to compose 
their own music/songs. Bass loops, drum loops, rhythm guitar and lead guitar riffs and 
30 loops in different musical instruments are provided. Samples are available on CDs and can 
be downloaded firom the Intemet. 



wo 2004/008430 



PCT/IE2003/000102 



7 

Preferably, the user will be able to set beats per minute BPM, create his own riffs, loops, 
and effects and change the pitch of selected sections. BPM stands for beats per miaute and 
is also known as the tempo of a song, or in other words the speed at which a song is 
played. Different songs will have different BPM e.g. a lot of Techno/Dance/Hip-Hop will 
have 130-180 BPM. It is important when creating a song made up of sample riffs and loops 
that all the samples have the same BPM. Some software programs allow the transposition 
of samples from one tempo to another without changing the pitch of the sample. 

Ideally, a number of digital musical instruments can be connected to the central control 
unit at one time allowing multi-user operation of the apparatus. One guitarist could control 
the lead guitar, another the bass, and another the rhythm guitar and roles could be switched 
while playing. In Jam mode, players could improvise by playing over specially composed 
songs or by playing their own tracks/songs or by playing in random selection mode. In this 
mode a number of operators could have a *battle of the bands' competition against each 
other. 

In another aspect of the invention, an operator uses the apparatus as a learning aid and has 
to strum to the correct tempo of the music as well as making different track selections and 
adding the proper effects at the rigjit time. 

In a further aspect of the invention, the apparatus is used as a controller to bring the 
operator through different levels of a custom designed computer game. The game plot 
could go through different levels of becomiag a rock star such as going to music school, 
learning to play, forming a band, writing songs, playing gigs, getting a manager, recording 
in a studio, getting a record deal, releasing an album, designing CD sleeves, makmg a 
pop/rock video, animations/cUps etc, competmg in the charts and all the various stages 
could be conducted as a competition over the Internet. 

Most guitar players write songs initially as a sequence of Chords. There are numerous 
music books available to give the guitar tablature (Chords) for different music 
albums/styles. These could also be provided from a website. 
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The present invention is a combination of digital hardware aad computer software 
program. It operates on mass market computer-based multimedia platforms, i.e. personal 
computers or games consoles such as Sony Playstation, Microsoft X-Box or Nintendo 
Dreamcast (APS Registered Trade Marks). 

The digital media content for use with the invention can be any third-party generated 
media that can be rendered on a personal computer or games console. 

The content also includes discrete soundbites (ejBfects sounds, i.e. riffs, beats, loops etc. 
selectable by the switches on the peripheral) that can be triggered during the playback 
experience. 

The invention will now be described with reference to the accompanying drawings, which 
show, by way of example only, an interactive multimedia apparatus in accordance with the 
invention in which: - 

Figure 1 is a pictorial representation of a first embodiment of interactive 
multimedia apparatus; 

Figure 2 is a pictorial representation of a second embodiment of the interactive 
multimedia apparatus; 

Figure 3 is a pictorial representation of a third embodiment of the interactive 
multimedia apparatus; 

Figure 4 is a perspective view of a digital musical instrument provided in this case 
by a guitar; 

Figure 5 is a front elevation view of the guitar of Figure 4; 
Figure 6 is a side view of the guitar of Figures 4 and 5; 
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Figure 7 is a rear exploded perspective view of the guitar of Figures 4, 5 and 6; 
Figure 8 is a schematic diagram of a USB (Universal Serial Bus) device; 
5 Figure 9 is a schematic diagram of the remaining section of USB device; 

Figure 10 is a menu of all C chord variations held in the store; 
Figure 11 is a screen shot showing the instrument selection folder; 

10 

Figure 12 is also a screen shot showing the instrument selection folder;. 

Figure 13 is a screen shot showing a control member functionaUty assignment 
screen; 

15 

Figure 14 is also a screen shot showing the control member functionality 
assignment screen; 

Figure 15 is also a screen shot showing a further control member functionality 
20 assigmnent screen; 

Figure 16 is a screen shot showing a special effects assignment screen; 

Figure 17 is a screen shot showing a further special effects assignment screen; 

25 

Figure 18 is a screen shot showing further options on the special effects assigmnent 
screen; 

Figure 19 is a further screen shot showing a special effects assignment screen 
30 assigning special effects to specific control member; 



wo 2004/008430 



PCT/IE2003/000102 



10 

Referring to the drawings and initially to Figure 1 there is shown an interactive multimedia 
apparatus indicated generally by the reference mimeral 1. The apparatus 1 comprises a 
central control unit provided by a PC 2 in this embodiment. The PC 2 can store any 
number of digital media in any desired file format on a hard drive and has I/O connections 
5 for a digital musical instrument provided by a guitar 3 in this embodiment. The PC 2 also 
has an I/O coimection for audio- visual equipment provided by an audio unit 4 and Internet 
access via a modem 5. Alternatively, the speakers 10, lOA and drive means (sound card) 
can form part of the PC 2. 

10 Referring to the drawings and now to Figure 2 there is shown a second embodiment of an 
interactive multimedia apparatus indicated generally by the reference numeral 11. The 
apparatus 1 1 comprises a central control unit provided by a set top box 12 having a facility 
to store any number of digital media in any desired iBle format downloaded over a cable 
TV modem 13, The central control unit 12 receives electrical signals from the guitar 14. In 

15 response to electrical signals from the guitar 14, the software of the central control unit 12 
generates an audible output signal from the digital media to the audio-visual equipment 15 
in direct response to an input from the txser of the guitar 14. 

Referring to the drawings and now to Figure 3 there is shown a third embodiment of an 
20 interactive multimedia apparatus indicated generally by the reference numeral 21. The 
apparatus 21 comprises a central control unit provided by a games console 22 having a 
facility to store any number of digital media in any desired file format downloaded over a 
modem 23. In response to electrical signals from the guitar 24, the software of the console 
22 generates an audible output signal from the digital media file to the audio-visual 
25 equipment 25 in direct response to input from the user of the guitar 24. 

Referring to the drawings and now to Figures 4 to 7, there is shown one embodiment of a 
digital musical instrument provided in this case by a guitar 41. The guitar 41 is provided 
with dynamic range state control members in the form of strings 42 and five transducers 50 
30 and Ught emittmg diodes (LED's) 52 disposed on the housing 43. Each transducer 
independently monitors a separate string 42. It will of course be appreciated that a 
separate transducer 50 is provided for each dynamic range state control member. A number 
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of limited state control members 44 are provided in addition to a dynamic range state 
control member in the form of a tremolo arm 45 for pitch, bending and creating a tremolo 
effect. A further dynamic range state control member in the form of a volume control 
button 46 and a power indicating L.E.D. 47 are also provided on the guitar 41. Also 
5 shown in Figures 5 to 7, a control panel 48 is provided at the end of the arm 49 of the 
guitar 41 and is provided by four limited stated control members, namely switches. 

In use, a user navigates through a G.U.L of the software suite on a VDU (15, 25) using the 
control panel 48 and assigns specific files stored on the store to one or more control 

10 members 42, 44, 45 46. Additionally, the user may apply various effects to either or both 
of the limited state control members 44 and the dynamic range state control members 42, 
45 46 which is more fiiUy explained with reference to Figures 10 to 19 below. Once file 
and effect assignments are made, the user strums the guitar 41 and each string 42 vibrates 
up through its ovm transducer disposed on the housing 43. The transducers convert the 

15 mechanical vibrations to an electrical signal and forwards the electrical signal to a USB 
interface unit (see Figures 8 and 9). The USB transmits the signals to the central control 
unit (2, 12, 22) and in particular to the software suite stored thereon. In response to 
electrical signals firom the guitar (3, 14, 24, 41), the software opens a file containing a 
variety of sounds/images stored digitally and produces an output via connected speakers or 

20 V.D.U's. 

If a user wishes to output sound from a different file, another Umited state control member 
44 must be pressed on the guitar 41. This in turn signals the software to open a different 
associated file. If a user wishes to apply a special effect they can activate any control 

25 member 42, 44, 45, 46 such as the arm 45 and volume control is achievable by twisting 
volume control buttons 46. When a user becomes tired of the various files that they have 
downloaded onto the central control unit (2, 12, 22), they may select a different collection 
of files using the control panel 48 in conjunction with a graphical user interface displayed 
on the visual display. A user can directly access the internet using the guitar 41 as a means 

30 for navigation. 
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Referring to the drawings and now to Figures 8 and 9, there is shown a control unit of the 
digital musical instrument, namely a USB (Universal Serial Bus) implementation of the 
interface hardware for the Interactive Multimedia apparatus (1, 11, 21), combined with a 
custom-designed suite of software running on a central control unit (2, 12, 22), which will 
5 provide the user with the faciUty to render the full range of instrumental chords by 
selection of pre-assigned control members (42, 44, 45, 46) and the simultaneous activation 
of an individual control member string (42) or a plurality of string control members (42). 
This invention allows non-musical trained users to compose, accompany, solo, gig and 
have fun as if they were an accomplished instrument player. 

10 

Additionally the invention will assist users in developing a knowledge of chord structures, 
complex chord structures, scales and the location of notes on the fret of a stringed 
instrument (3, 14, 24, 41). 

15 This invention allows users to not only dynamically apply global effect parameters to a 
selected note or chord, but will allow users to select, control and adjust any individual or 
group of parameters which make-up the component parameters of the selected special 
effect generator. 

20 Figure 8 and Figure 9 show the schematic drawings for the USB (Universal Serial Bus) 
Interactive Multimedia apparatus (1, 11, 21). The explanation of the circuit and its 
operation are as follows: 

The guitar (3, 14, 24, 41) product is a USB low speed (1.5 Mhz) bus powered device. It has 
25 12 pushbutton switches SWl to SW12, 1 optoswitch OPTl/red LED 204SRC pair. It is 
coimected to the PC via a 3m 4 core screened cable. 

Schematic Description 
Power 

30 The guitar (3, 14, 24, 41) product receives its +5 volts power from the PC via USB 
connector CNl. The maximum current drawn will be approximately 50mA. 
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Hardware reset 

When power is first applied, the CPU will be reset by 2 off O.luF capacitors and 10k 
resistor combination. Suspend Mode, AH USB devices must support suspend mode. 
Suspend mode enables the device to enter a low power mode if no activity is detected for 
5 more than 3mS. Any bus activity will keep the device out of the suspend state. When the 
device is in suspend mode it must draw less than 500uA. CPU Ports A and C are 
configured as outputs, and set to high, when entering suspend mode because as inputs each 
pin of ports A and C will draw 50uA due to the internal pull-up resistors on these ports. 
CPU Port B does not contain any intemal pull-up resistors but external pull-up resistors are 
10 implemented in hardware at the optocoupler phototransistor output. Thus all port B CPU 
pins should be configured as outputs and set to high applied before entering Suspend 
mode. Disabling the Analogue to Digital converter will save some current in suspend 
mode. In suspend mode the current drawn by the guitar (3, 14, 24, 41) must be less than 
500uA. 

15 

Entering suspend mode 

1 . Make PAl output high 

2. Malce PA2 output high 

3. Make the following port pins and outputs HIGH 
20 PA0,PA3,PA4,PA5,PA6J>A7,PB0J*B1J>B2,PB3,PB5,PB6,PB7J>C0,PC1J*C2 

Note: leave PB4 as an input always 

Exiting suspend mode 

25 The product can be woken up from suspend mode by switching the bus state to the resume 
state, by normal bus activity, by signalling a reset or by an external interrupt. During 
suspend mode the intemal CPU oscillator is turned off. In this state the CPU will not be 
able to detect key presses. 
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Leaving suspend mode 

1 . Make the following port pins Inputs 

PA0,PA3,PA4,PA5,PA6,PA7,PB0,PB1,PB2JPB3,PB4J>B5,PB6J>B7,PC0,PC1,PC2 2. 
Make PA2 output LOW 
5 3. Malce PAl output LOW 

Guitar (3, 14, 24, 41) will only be in suspend mode:- 

a) Wlien not configured in tiie PC 

b) When told to do so by the PC 

c) When tiiere is no bus activity 

10 

CPU(U1) 

The CPU (Ul) is a ST7263 manufactured by ST Microelectronics. The CPU version is a 
surface mounted type called ST72T631K4M1. The CPU clock is set by a 24 MHz crystal 
(A) 

15 

Switches SWl to SW12 

There are a total of 12 push button switches (normally open) connected to the CPU. Each 
switch is monitored by 1 separate input. Each input is joined to +5 volts via a pull-up 
resistor. When a switch is pressed the input will drop j&om +5 volts to 0 volts. 

20 

Opto Svidtch OPTl and LEDl 

OPTl is a phototransistor which is switched on when LEDl (red LED) is on. 

LEDl shines light on OPTl. OPTl is switched off by cutting the light beam with your 

thumb. 

25 

U2 

This IC protects the CPU from any spurious signals picked up by the external wure. 
USB004 is a Chinese reconomended part, 

30 U3 

This IC is a low power Op amp TS931 ILT. (ST Microelectronics). It is used to amplify 
the signal from the magnetic pickup. 
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Jl and J2 (3.5mm Jack sockets) 

Jl or J2 allow for an input from a standard variable resistor foot pedal. The middle 
coimection is the wiper of the potentiometer. A typical value for this potentiometer is 
5 approx. 22 k ohms 

Bill of Materials 





Descrintion 




1 


SMT resistor 1.5 ohm 


0.125w 5% tolerance 


1 


SMT resistor 150 ohm 


0.25 w 5% tolerance 


5 


SMT resistor 330 ohm 


0.125W 5% tolerance 


3 


SMT resistor lk5 


0.125w 5% tolerance 


14 


SMT resistor 4k7 


0.125w 5% tolerance 


2 


SMT resistor 10k 


0.125W 5% tolerance 


1 


SMT resistor 22k 


0.125W 5% tolerance 


1 


SMT resistor 150k 


0.125W 5% tolerance 


2 


SMT resistor 220k 


0,125w 5% tolerance 


1 


SMT resistor IMohm 


0.125W 5% tolerance 



20 

1 47k potentiometer (VRl) 10% tolerance 

2 SMT capacitor 33pF ceramic 50v 5% tolerance 

5 SMT capacitor 0. luF ceranaic 23v 10% tolerance 

25 1 SMT c^acitor 0.47uF ceramic 16v 10% tolerance 

2 SMT c£^acitor 1 OuF 1 6v tantulum or aluminium electrolytic 

1 RedLED204SRC/E 

1 opto transistor (0PT1)PT928-6C 

30 

1 SMT 24mHz crystal AELCM309S 
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2 1N914 diodes 

1 CPU (Ul) ST Microelectronics ST72F63BK4M1 
(SMT Shrink package S034 ,300niil width) 
5 1 protection chip (U2)USB004 

(SMT package) 
1 op amp TS93 1 ILT (ST Microelectronics) 

1 2 SMT switches (normally open) Omron B3S-1 000 

10 

5 Trauducers 

1 USB screened cable Smetres long with series 'A' Plug 



15 Port No.s for the Guitar 





PORT 


PdmNo. 

(PACKAGE IS 

S034) 


INTERR 
UPT 


Internal 

PULLUP 

Resistor 


Type of 
Signal 


Comment 


4 Switches at 
bottom 

(Bottom is SWl , 
Top is SW4 














SWl 


PAO 


29(Input) 


no 


puUup res. 


HiorLo 


0 means key is 
pressed 


SW2 


PAS 


23(Input) 


no 


pullup res. 


HiorLo 


0 means key is 

pressed 


SW3 


PA4 


22(Input) 


Ext. Int 

Loto 

Hi 


puUup res. 


HiorLo 


0 means key is 
pressed 


SW4 


PAS 


21(Input) 


Ext. Int 

Loto 

Hi 


pullup res. 


HiorLo 


0 means key is 
pressed 


8 Switches on neck 
Bottom is SW5 
top is SW12 














SW5 


PA6 


20(Input) 


Ext. Int 
Hi to 
Lo 


pullup res. 


Hi or Lo 


0 means key is 
pressed 


SW6 


PA7 


19(Input) 


Ext. Int 
Hi to 
Lo 


pullup res. 


HiorLo 


0 means key is 
pressed 
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SW7 


PB2 


16(Input) 


no 


no 


Hi orLo 


0 means key is 
pressed 


SW8 


PB7 


10(Input) 


Ext. lat 
Hi to 
Lo 


no 


HiorLo 


0 means key is 
pressed 


SW9 


PCO 


70nput) 


no 


puUup res. 


IfiorLo 


0 means key is 
pressed 


SWIO 


PCI 


6(Input) 


no 


puUup res. 


Hi or Lo 


0 means key is 
pressed 


SWll 


PC2 


S(Input) 








0 means key is 
pressed 


SW12 


PB6 


11 (Input) 


Ext.Int 
Hi to 
Lo 


no 


Hi or Lo 


0 means key is 
pressed 
















Ground pin for all 
12 Switches and 
opto 
device 

SWI to SW12 and 
optl 


PAl 


28 (Output) 
(25mA port) 


no 


p\iUup res. 


Hi or Lo 


Set to 0 to allow 
all switches SWI 
to SWI 2 and 
optol to be 
detected 
Set to 1 in 
suspend mode 
















RedLED2for 
OPTl 


PA2 


24 (Output) 
(25mA port) 


no 


no 


HiorLo 


Set to 0 to light 
LED 


OPTl 


PBO 


18([nput) 


no 


no 


Hi orLo 


1 means light is 
blocked 
















VRl pot (Wow 
pot) 


PB3 


15(Input) 


no 


no 


analog 




From Amplifier 
U3 


PB4 


14 (Iq)ut) 


Ext. Int 

Loto 

Hi 


no 


analog 

i/p 




S.Snun Jack socket 
(Jl) for Foot pedal 
1 


PBl 


17 (Input) 


no 


no 


analog 

i/p 




S.Smm Jack socket 
(J2) for Foot pedal 
2 


PBS 


12( Input) 


Ext. Int 

Loto 

Hi 


no 


analog 

i/p 





The above schematics and explanation are shown for example purposes only and the 
invention is not limited in its scope by the operation of the schematic design, the 
components used or the specification or capabilities of the components or the range of 
additional peripheral devices which could be added to the design. The design is not limited 
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to the number of control members (switches) or magnetic pick-ups, hghts etc. shown in 
these schematics. 

Particular attention should be drawn to Figure 9, Label K. The schematic shows the 
5 circuit for a standard multi-pole transducer which provides a single output energy source 
irrespective of the fact that one or any combination of strings 42 are activated. This 
invention uniquely specifies the use of a pluraUty of individual, xmcoupled, isolated 
transducers that will only be energised by the activation of the individual string 42 directly 
associated with that string. 

10 

Each transducer output would be connected to a separate pin on the CPU, in a similar 
fashion to that described in the schematic Figure 8 and Figure 9, which are shown as 
example only. The CPU would perform an analogue to digital conversion (A/D) on the 
sampled input. In the situation where sufficient A/D resources are not available on the 

15 CPU an external A/D converter would be required. Another alternative would be to bank 
switch groups of inputs, be they analogue (e.g. Transducer output) or digital (e.g. Push 
button). For example, assuming there is a requirement to read 16 analogue inputs, these 
inputs could be coimected to a multiplexing device whose 8 output pins are connected to 
the CPU. Another pin firom the CPU would control (banlc switch) which bank of 8 inputs 

20 are directed to the CPU. 

The two foot-pedal jack sockets shown in Figure 9 and described in the schematic 
workings will allow users to dynamically modify a variable control, for example volume, 
pan or special effect parameters controls as described later in this appUcation. 

25 The individual transducers (magnetic pick-ups) are one method of detecting string 42 
vibrations. The invention is not limited to the use of any specific tranducer type such as 
piezo crystal, piezo film, piezo ceramic opto-detection methods, pressure sensing, 
movement detection et al. could also provide acceptable solutions for sensing the 
vibrations, and amplitude of the vibrations. The preferred type of transducer is a piezo- 

30 electric transducer selected firom a group consisting of piezo-electric crystal transducers, 
piezo-electric ceramic transducers and piezo-electric film transducers. 
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Preferably the Interactive multimedia apparatus, see Figui'e 5 as an example, would include 
a series of coloured LED lights under each string 42, located at the neck end. The software 
activates the Ughts under the individual strings 42 which make up the selected chord, so 
that the user will be notified of the correct strings 42 to strum and if they so wish, they can 
5 pick the individual notes of the selected chord. This component of the application Avill be 
dealt with in more detail later in the application. 

In use, the invention may be as follows; 

10 Usually chords are played on a stringed instrument by the placement of the user's fingers 
on the selected strings 42 in the appropriate section of the fret on the neck piece and then 
the user activates the strings 42 in a manner that produces the desired sound output with 
the corresponding vibration amplitude and duration. The user may strum, pick, strum up, 
strum down, strum and pick, pick between selected strings 42 etc. to achieve their desured 

15 resxilts. The correct fingering position on the fret for each string 42 is a critical component 
in the generation of each note in the overall chord structure. 

Asian Pacific, European and USA music compositions use 3 note chords extensively, with 
the more accomplished, creative and dynamic instrumentalists using more complex chord 
20 combinations of 4, 5 and 6 notes to provide a more complete and colourfiil signature to 
their playing. 3 note chords can also have many combinations of individual notes from 
within its own scale, which provide the user with a rich palate of chords from which to 
select. 

25 If we take as an example only, the 'C Chord* and referring to Fig. 10; 
The diatonic C scale is as follows: 

CC* DD*EFF*GG*AA*BC 

J)b gb Qh gb 

30 

is the same note as C* and the other 6(flat) notes are the equivalent of the # (sharp) 
notes. 
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The three note chords or triads for the C scale are C, E, G; for Cm they are C, , G , for 
CSus4 they are C, F, G,; for Cm65 they are C, E^ , G^ ; for C+ they are C, E, G'^ . It is 
obvious that for each note there is a multiplicity of chord associations for the user to create 
or select from. It is nearly hnpossible for the non-accompUshed user to create the more 
5 complex chords, particularly those using 4, 5 or 6 note combinations. 

Many users with limited experience and trainiag have been able to create simple 
compositions using 3 chords variations. They can use the chords of the root note of that 
key and the chords of the fourth and fifth notes of the scale of the root note. Limiting the 

10 creative and entertaining experience to 3 or a small number of chord combinations is most 
finstrating and irritating to users. To progress beyond the simple "3 chord trick" 
combination requires a lot of learning and practice with fret fingering etc. This invention 
eliminates the complexity of correct fret fingering, simple chord creation, complex chord 
creation, learning note combinations associated for simple and complex chords etc. 

15 Additionally this invention will quickly teach the user if they wish to learn, the correct 
notes associated with each chord they select and moreover fiie user is shown which string 
42 has triggered each note of the selected chord. 

This invention allows users to quickly create and render an unlimited number of 
20 combinations of simple and complex chords, where each note of the chord is rendered at its 
correct interval in the chord structure as it would be played by an accomplished user of a 
stringed instrument. 

Additionally, this invention will allow the user to play complex arpeggios as the notes 
25 associated with each arpeggio are stored on file and are assigned by the software to the 
correct string position. 

The chord generation and note rendering methodology for string-based instruments and 
that of keyboard-based instruments differ greatly. With keyboard-based instruments, the 
30 user usually plays a chord by simultaneously depressing the keys of the associated notes of 
the chord. It is not normally possible for the user of a stringed instrument to strum or to 
pick each note of a chord simultaneously whilst fingering the notes on the fret. This is 
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because there is a time difference interval between the triggering of each note of the chord. 
This invention allows for this tune difference mterval, as the chord store contains the 
correct location of each note of the selected chord and tiien assigns each note to the correct 
string position on the mteractive multunedia apparatus. Therefore, as each individual 
5 transducer's energy level is detected for each string activation, the individual notes of the 
chords are rendered as the correspondmg string 42 is activated. In this way the user is 
provided with a true representation of an accompUshed player's experience. 

The LED visual displays under each string 42 will be energised by the appUcation software 
10 to indicate to the user the active' strings 42 for the selected notes of the chord. The 
provision of the LED display is to allow users to visibly see the different note assignments 
to each string for each chord combination. The software can apply appropriate notes to 
the unassigned strings 42 to simulate the sounds that these unassigned strings 42 would 
make if the user wishes to strike across the whole of the string area in a violent action 
15 similar to the action of their hard-rock idols. The user can chose from a set-up menu, 
whether they wish to configure the device for violent actions and have the software 
automatically trigger the correct sounds for the unassigned strings 42. 

The activation of the strings 42 resultmg from the user actions will be interpreted by the 
20 software to produce a sound output that truly reflects the user's striking actions. 

The user experiences described above are achieved usmg the techniques described as 
follows: 

In this embodiment, the Interactive Multimedia apparatus (1, 11, 21) has 12 control 
25 members, refer to Figure 5, 44, which shows only 4 control members 44 as an example 
only, and any number of additional control members 44 for other assignments. 

The transducer specified m this appUcation will comprise five in number (as example only 
and not Umited to this number) mdividual, uncoupled and electrically isolated transducers, 
30 which provides for five separate electrical outputs as a response to the user's activation of 
the individual string 42. 
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For example purposes only we will describe how the user will interact with the software to 
simulate similar results to those achieved by an accomplished instrumentaUst. 

Accordingly there is provided a store of recorded instrument notes from a pluraUty of 
5 stringed instrument types. A chord store is provided which contains the note associations 
for a very wide range of defined chord structures. The users, if they so wish, can 
additionally create their own bespoke chords or note associations, by selecting an 
association of notes and defining their string associations. 

10 This chord store will provide the note associations for a very wide selection of known 
chord and scale definitions and structures. To simulate the playing of a stringed instrument 
using the Interactive multimedia apparatus (1, 11, 21) and its associated software 
application program, demands the faithful reproduction of the selected instrument sound in 
complete and total harmony and sympathy with the user's actions. It is imperative that the 

15 chord store contains the exact string assignment for each note in the chord so that the 
action of strumming or picking the chord will trigger the notes in the correct order that they 
were played by the user. In this invention, the store which has been created contains the 
note associations and the string assignments for each chord contained in the store. 
Additionally, by user selection, the chord store may assign complimentary and sympathetic 

20 notes to any or all of the unassigned strings 42. The assignment of sympathetic notes to 
imassigned strings 42, will provide a more colourfiil and verbose chord rendering. 

Accordingly, if the user wishes to render a chord, for example the chord 'C, the user 
would firstly select the instrument file as shown in Figure 1 1 Label A and then select the 

25 chord 'C from the chord store for the selected instrument type, see Figure 10 Label A. The 
user would then be presented with a drop down menu of all the C chord variations held in 
the store, for example purposes only see Figure 10 Label A. The user may wish to select 
the major chord. Figure 10 Label B, which contains the notes C, E, G. The chosen chord 
must then be assigned to a control member, see Figure 5, reference numeral 44, which will 

30 select that chord when the user activates that control member 44. The chord assigned to 
that control member 44 will be rendered by the software when the user activates the strings 
42 of the Interactive Multimedia apparatus (1, 1, 21). The software will have automatically 
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assigned the notes of the selected chord to the appropriate strings 42 of the interactive 
multimedia apparatus (1, 11, 21) so that the individual transducer associated with each 
string 42 will detect the user's actions in striking the string 42 and provide an audio output 
for the associated assigned note that will faithfully represent the response to the user's 
5 actions. 

String assignments for the Simple D major chord would be String 1, Note F*^; String 2, 
Note D and String 3, Note A. If the user has pre-selected the option to apply an additional 
note, then the software will add a further note 'D' to the 4*^ String. 

10 

The user can assign chords and special effects to the control members 44, which for 
example purposes only will be illustrated as follows; the user will select from a window, 
see Figure 12 Label A, the particular chord they wish to assign to a control member 44. 
The user is then presented with a menu as shown in Figure 13. The user must select the 

15 device type from a plurality of devices, as shown in Figure 13 Label A. For example 
purposes only we show a guitar-type device called the TikAx' in Figure 13 Label A. The 
user must then select the device number they wish to be configured, see Figure 13 Label B. 
This appHcation allows for a plurality of devices in each class of device and for a pluraUty 
of different device classes. The user will then select from a drop-down menu, see Figure 13 

20 label C, containing various control member identifier options. The user specifies which 
control member they wish to assign the selected chord to. For example purposes only, we 
show in Figure 13 Label C, the control member being identified as Switch 10, See Figure 
5, reference numeral 44. The user will then select the electrical state the control member 44 
must reach to be in the state of assertion for the selection of the assigned chord, as shown 

25 for example purposes only in Figure 14 Label A. The drop-dovra menu as show in Figure 
14 Label A illustrates for example purposes only, four conditions of assertion - *when 
pressed', *when released', 'while pressed' and *while released'. The user will then select 
the triggering method for the control member, see Figure 14 Label B, which when asserted 
will provide the stimulus to the software to render the notes of the chord in direct response 

30 to the software's interpretation of the control members vibrations or activations. In some 
device class types, the activation source may not be a vibrating mechanism, but some other 
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pressure sensing device, an opto-coupled device or any other transducer, which provides an 
electrical output in response to a user's actions. 

In this example we show in Figure 14 Label B that a vibration sensitive transducer is the 
5 selected activation device for triggering the notes of the assigned chord. The user may wish 
to adjust various effects, controls, characteristics or parameters of the soimd output by 
using either a fixed or adjustable control member (42, 44, 45, 46). In Figure 15, we show 
for example purposes only, a range of controls that the user may wish to adjust 
dynamically by using one of the dynamic range state control members (42, 45, 46) shown 

10 in Figure 19 Label A or by using any of the limited state control members 44 shown in 
Figure 13 Label C. The user may wish to increase the output volume, see Figure 15 Label 
A; the pan control, see Figure 15 Label B; the tempo, see Figure 15 Label C or any other 
control or parameter they may consider is desirable to adjust dynamically. In this example, 
for a limited sate control member 44, the user will assign the selected control member 44 

15 for the control adjustment by selecting from the drop-down menu, Figure 15 Label B and 
then selecting the assertion state for that control member Figure 15, Label E and then 
selecting the percentage adjustment they wish to apply, Figure 15 Label F and also the rate 
of change in milliseconds at which they wish to apply the change, Figure 15 Label G. 
Similarly to reduce the volume, pan, and tempo the user selects and assigns the controls as 

20 shown in Figure 15 Label H, J, K, L. 

When using an adjustable control member (42, 45, 46), the user will select and assign the 
appropriate control members. Figure 15, Label M, N, P. The user may wish to mute the 
output by selecting and assigning the control member and their assertion states, see Figure 

25 15 Label R and S. The user may wish to restore the original settings by selecting and 
assigning a control member, see Figure 15 Label T and V. Another significant and unique 
component of this application is the abihty of the software to dynamically and in real time, 
adjusts the individual parameters of a special effect, which is being applied to an individual 
note or notes of the selected chord, while it is being rendered by the central control unit. In 

30 this application and for example purposes only, we show in Figure 16 Label A, a range of 
special effect choices firom which the user may select and which they may apply 
dynamically to a chord or note or a combination of notes. In this example, we show that 
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the user has selected *flanger', see Figure 16 Label B. Figure 17 shows, as example only, 
some of the key parameter adjustments that affect the generation of the 'flanger' special 
effect. The user may adjust any or all of these parameters to provide a composite sound 
effect of their choice. Additionally the user may wish to dynamically adjust in real-time 
5 during their playing activity any or all of the individual parameters to create contrasting 
sound effects. The user sets the individual slider controls for each parameter as in Figure 
17 to provide the composite sound effect results they desire. The user will then assign the 
parameters they wish to adjust dynamically and in real-time as follows; the user wishes to 
adjust a parameter of the *flanger' special effect and to have the selected parameter 

10 irmnediately activated in response to their operation of an adjustable control member. (42, 
45, 46) Figure 18 Label A shows a selection box, which when selected will tell the 
software to automatically apply the adjustments when the selected control member is 
activated. Figure 18 Label B shows a drop-down menu containing the individual 
parameters of the special elBfect being applied as in Figure 17. The user selects the 

15 individual parameter they wish to dynamically adjust and in this example it is the 
frequency. The user will then assign the adjustable control member they desire as the 
trigger mechanism, see Figure 19 Label A. In this example, it is a foot-pedal. Additionally 
there is allowance, as in Figure 19 Label B, for the user to select a minimum threshold 
level before the adjustable control member (42, 45, 46) kicks in. Setting this minimum 

20 threshold level in the activation of the parameter adjustment allows for differing tolerances 
in the electrical properties of proportional potentiometers and other proportional 
measurement devices. 

When the user activates the selected control member (42, 44, 45, 46), which in this 
25 example is a foot-pedal, beyond the minimum threshold level of 5%, then the software will 
adjust the frequency parameter of the effect in sympathy with the movement of the 
dynamic range state control member (42, 45, 46). 

Additionally there is provided ftulher embodiments of this invention, which provides an 
30 entertaining, but less challenging and less educational experience for the user. In this 
ftirther embodiment, the chords would be rendered by the software in response to the user 
striking any of the strings 42 of the interactive multimedia apparatus (1, 11, 21), in any 
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sequence of struniming, picking etc. In this further embodiment, the notes of the chords are 
not rendered as a response to the individual string 42 activation as described in the earlier 
embodiment. This embodiment may be a starting position of choice for beginners, who 
would quickly progress to the more advanced embodiment as described earlier in this 
5 application. In this embodiment, the user must select the preferred tempo they wish to use. 
The software will trigger the rendering of the notes of the selected chord at time intervals 
appropriate to the selected tempo. 

Additionally, the chord store is not restricted to the notes of a variety of string instrument 
10 types. The chord store could include notes, chords or sounds from any instrument type or 
from any percussion type instrument or from any wind-based instrument or from any reed- 
based instrument or from any instrument that is activated by a bow movement or from any 
device that comes within the classification of a musical instrument, which would be 
rendered as either individual notes, chords or sounds or combination of sounds by user 
15 selection. This invention is not limited in any way by the examples provided in this 
apphcation or to the instrument types or to the method of application or stimulation. 

It is to be understood that the invention is not limited to the specific details described 
above which are given by way of example only and that various modifications and 
20 alterations are possible without departing from the invention as defined in the appended 
claims. 



